feat: upgrade @mcp-ui/client package and improve UI message handling#4164
feat: upgrade @mcp-ui/client package and improve UI message handling#4164
@mcp-ui/client package and improve UI message handling#4164Conversation
|
.bundle |
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
…nd package-lock.json
…o use isUIResource utility
…ror management - Introduced specific result types for tool calls, prompts, links, notifications, and intents. - Added error handling with strongly typed error codes for better clarity and debugging. - Implemented separate handlers for each action type to improve type safety and maintainability. - Updated the main action handler to support new action types and provide exhaustive type checking.
- Implemented a new IPC handler in the main process to open external URLs securely. - Updated the preload script to expose the openExternal function for invoking the new handler from the renderer process.
…sage handling - Added a global event listener for scroll-to-bottom requests in BaseChat to improve user experience. - Enhanced MCPUIResourceRenderer and ToolCallWithResponse components to support an append function for message handling. - Updated prompt action handling to utilize the append function, providing fallback options for message delivery.
… and enhancing error handling - Removed optional callbacks for tool calls, prompts, navigation, and intents to streamline the component. - Improved error handling for unsupported actions in tool calls, prompts, and intents. - Ensured that the append function is utilized for prompt actions and added custom event dispatching for chat scrolling.
…message handling - Renamed the append prop to appendPromptToChat for clarity in MCPUIResourceRenderer. - Updated ToolCallWithResponse to pass the new appendPromptToChat prop to MCPUIResourceRenderer. - Revised TODO comments for better clarity on future enhancements regarding message handling.
…dling - Expanded action handling in MCPUIResourceRenderer to include specific cases for tools, prompts, links, notifications, and intents. - Improved error handling with consistent status codes and messages for unsupported actions. - Streamlined the main action handler for better readability and maintainability, ensuring exhaustive type checking. - Removed unused callback functions to simplify the component structure.
…nous - Converted action handler functions in MCPUIResourceRenderer to asynchronous to support promise-based operations. - Ensured that the main action handler awaits results from tool, prompt, notify, and intent cases for improved error handling and response management.
| const handleUIAction = async (actionEvent: UIActionResult): Promise<UIActionHandlerResult> => { | ||
| console.log('[MCP-UI] Action received:', actionEvent); | ||
|
|
||
| case 'notify': { | ||
| // TODO: Implement notify handling | ||
| handleAction(result); | ||
| break; | ||
| let result: UIActionHandlerResult; | ||
|
|
||
| const handleToolCase = async ( | ||
| actionEvent: UIActionResultToolCall | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { toolName, params } = actionEvent.payload; | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Tool calls are not yet implemented', | ||
| details: { toolName, params }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| const handlePromptCase = async ( | ||
| actionEvent: UIActionResultPrompt | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { prompt } = actionEvent.payload; | ||
|
|
||
| if (appendPromptToChat) { | ||
| try { | ||
| appendPromptToChat(prompt); | ||
| window.dispatchEvent(new CustomEvent('scroll-chat-to-bottom')); | ||
| return { | ||
| status: 'success' as const, | ||
| message: 'Prompt sent to chat successfully', | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.PROMPT_FAILED, | ||
| message: 'Failed to send prompt to chat', | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| case 'prompt': { | ||
| // TODO: Implement prompt handling | ||
| handleAction(result); | ||
| break; | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Prompt handling is not implemented - append prop is required', | ||
| details: { prompt }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| const handleLinkCase = async (actionEvent: UIActionResultLink) => { | ||
| const { url } = actionEvent.payload; | ||
|
|
||
| try { | ||
| const urlObj = new URL(url); | ||
| if (!['http:', 'https:'].includes(urlObj.protocol)) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Blocked potentially unsafe URL protocol: ${urlObj.protocol}`, | ||
| details: { url, protocol: urlObj.protocol }, | ||
| }, | ||
| }; | ||
| } | ||
|
|
||
| await window.electron.openExternal(url); | ||
| return { | ||
| status: 'success' as const, | ||
| message: `Opened ${url} in default browser`, | ||
| }; | ||
| } catch (error) { | ||
| if (error instanceof TypeError && error.message.includes('Invalid URL')) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.INVALID_PARAMS, | ||
| message: `Invalid URL format: ${url}`, | ||
| details: { url, error: error.message }, | ||
| }, | ||
| }; | ||
| } else if (error instanceof Error && error.message.includes('Failed to open')) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Failed to open URL in default browser`, | ||
| details: { url, error: error.message }, | ||
| }, | ||
| }; | ||
| } else { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Unexpected error opening URL: ${url}`, | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
| }; | ||
|
|
||
| case 'tool': { | ||
| // TODO: Implement tool call handling | ||
| handleAction(result); | ||
| break; | ||
| const handleNotifyCase = async ( | ||
| actionEvent: UIActionResultNotification | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { message } = actionEvent.payload; | ||
|
|
||
| try { | ||
| const notificationId = `notify-${Date.now()}`; | ||
| toast.info(message); | ||
| return { | ||
| status: 'success' as const, | ||
| data: { | ||
| notificationId, | ||
| displayedAt: new Date().toISOString(), | ||
| message, | ||
| }, | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: 'Failed to display notification', | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| }; | ||
|
|
||
| const handleIntentCase = async ( | ||
| actionEvent: UIActionResultIntent | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { intent, params } = actionEvent.payload; | ||
|
|
||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Intent handling is not yet implemented', | ||
| details: { intent, params }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| try { | ||
| switch (actionEvent.type) { | ||
| case 'tool': | ||
| result = await handleToolCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'prompt': | ||
| result = await handlePromptCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'link': | ||
| result = await handleLinkCase(actionEvent); | ||
| break; | ||
|
|
||
| default: { | ||
| console.warn('unsupported message sent from MCP-UI:', result); | ||
| break; | ||
| case 'notify': | ||
| result = await handleNotifyCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'intent': | ||
| result = await handleIntentCase(actionEvent); | ||
| break; | ||
|
|
||
| default: { | ||
| // TypeScript exhaustiveness check | ||
| const _exhaustiveCheck: never = actionEvent; | ||
| console.error('Unhandled action type:', _exhaustiveCheck); | ||
| result = { | ||
| status: 'error', | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: `Unknown action type`, | ||
| details: actionEvent, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
| } catch (error) { | ||
| console.error('[MCP-UI] Unexpected error:', error); | ||
| result = { | ||
| status: 'error', | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: 'An unexpected error occurred', | ||
| details: error instanceof Error ? error.stack : error, | ||
| }, | ||
| }; | ||
| } | ||
|
|
||
| // Log result with appropriate level | ||
| if (result.status === 'error') { | ||
| console.error('[MCP-UI] Action failed:', result); | ||
| } else { | ||
| console.log('[MCP-UI] Action succeeded:', result); | ||
| } | ||
| }, []); | ||
|
|
||
| return result; | ||
| }; |
There was a problem hiding this comment.
Hey @idosal, thanks again for taking a look at this.
I took another pass at handling UI actions here. Now, I'm returning a lot more info. Is this more in line with your suggestion?
There was a problem hiding this comment.
Thanks @aharvard, it looks great. It captures the data that could be relevant to the UI.
This dependency was accidentally included through a merge conflict resolution and should not be there.
@mcp-ui/client package and start UI message implementation
@mcp-ui/client package and start UI message implementation @mcp-ui/client package and improve UI message handling
|
.bundle |
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
| toolResponse={toolResponsesMap.get(toolRequest.id)} | ||
| notifications={toolCallNotifications.get(toolRequest.id)} | ||
| isStreamingMessage={isStreaming} | ||
| append={append} |
There was a problem hiding this comment.
append is an existing convention in this file, and I had to add it to the <ToolCallWithResponse /> component to prop drill. TBH, this feels like a hack.
Inside of <ToolCallWithResponse />, append is passed to <MCPUIResourceRenderer content={content} appendPromptToChat={append} /> — where, I think, appendPromptToChat is a bit clearer.
The goal is to
- Catch the prompt message from an MCP UI
- Pass that message to the chat engine
- Let goose take the wheel
@zane, do you know if there is a more elegant way to pass a prompt string to the chat engine w/out prop drilling?
There was a problem hiding this comment.
I think prop drilling is ok for 2 levels like this for now but we don't have any other global state mechanism currently other than React context. So you could use context if needed. We plan on adding a global state library soon that can make things like this easier.
| }); | ||
|
|
||
| // Handle external URL opening | ||
| ipcMain.handle('open-external', async (_event, url: string) => { |
There was a problem hiding this comment.
what functionality does this add that wasn't before - is it for cases where the type is links and it will open it in external browser? (if so - a hyperlink won't do that?)
There was a problem hiding this comment.
Since MCP-UI hyperlinks live inside of an iframe, clicking on a typical <a href="" /> link will refresh the contents inside of the iframe. That might be perfect for some MCP-UIs but not others.
Also, I think that Electron brings in some form of protection that may prevent hyperlinks from performing as authored, to protect the app user. (I have a lack of knowledge in this area, I could be wrong)
For example, in a typical web browser that has an iframe on the page, the hyperlink author can add an attribute to force a navigation at the parent level. In our Electron app, these attributes do nothing and clicking on a link fails to do what a user might expect it to.
The MCP-UI SDK gives us the link message type for, I believe, better security hygiene. So adding open-external was my attempt at finding the safest execution route.
|
I think a good upgrade - main Q was what to test for expecially adding messages, and also - the new open rpc - not sure what that does that a hyperlink can't do? |
…and toast notifications - Introduced new action types for size changes, iframe readiness, and data requests to improve message handling. - Implemented a ToastComponent for displaying notifications with support for implemented and unimplemented message types. - Updated the main action handler to accommodate new action cases, ensuring comprehensive handling of messages from the iframe. - Improved theme management for toast notifications to enhance user experience.
|
FYI, just push some last bits of feature refinement. Ready for re-review @zanesq, @michaelneale, @JHKennedy4, and others. TY! |
|
.bundle |
| "overrides": { | ||
| "react": "^19.1.1", | ||
| "react-dom": "^19.1.1" | ||
| }, |
There was a problem hiding this comment.
had to do this, but we can remove when this issue is completely resolved MCP-UI-Org/mcp-ui#90
|
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
zanesq
left a comment
There was a problem hiding this comment.
LGTM after react version issue fixed
* main: (108 commits) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) Enhanced onboarding page visual design (#4156) feat: adds mtls to all providers (#2794) (#2799) Don't show a confirm dialog for quitting (#4225) Fix: Missing smart_approve in CLI /mode help text and error message (#4132) ...
* main: docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235)
* main: (42 commits) feat: Add message queue system with interruption handling (#4179) Start extensions concurrently (#4234) Add X-Title and referer headers on exchange to tetrate (#4250) docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) ...
block#4164) Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
block#4164) Signed-off-by: Dorien Koelemeijer <dkoelemeijer@squareup.com>
This PR enhances MCP-UI functionality in Goose.
Chore: MCP-UI Upgrade
I bumped @mcp-ui/client to version (5.9.0), this lets us experiment with the following:
htmlProps.sandboxPermissionsisUIResource(content)helper for convenienceFeat: Allow forms for iframe sandbox
We got a request to
allow-formsfrom our MCP-UI iframe: #4117. So, this PR allows us to experiment with settinghtmlProps.sandboxPermissionstoallow-forms— for more info refer to docs https://mcpui.dev/guide/client/resource-renderer#props-details.Important
I'm unsure what attack vectors this introduces but would like to ask if we can be flexible and quick to make updates to
htmlProps.sandboxPermissionsover time as we learn more.Feat: Embedded UI Message Support
For more info on what MCP-UIs can message to Goose, refer to: https://mcpui.dev/guide/embeddable-ui#message-types
goose-mcp-ui-message-handling.mov
openExternal()method.Fix: UI Action callback response
In our first iteration,
onUIAction={handleUIAction}did not return a meaningful response (refer to this issue raised over on mcp-ui). Now it does.